posted 05-29-2011 04:56 AM            

APA
"14.7 To maximize the informational efficiency of multiple issue (mixed-issue) testing and the diagnostic efficiency of single-issue screening exams, a successive hurdles approach is recommended."

1. Is there difference in accuracy between single-issue test with known allegation(event) and single-issue test with unknown allegation(event)? Are they both diagnostic tests?

2. Is there difference in accuracy between standalone screening single-issue test (without prior multiple issue testing) and single-issue test with prior multiple issue testing (successive hurdles approach).
Are they both diagnostic tests?
Are they both with unknown allegations?

Thanks.

posted 05-29-2011 04:27 PM            

It is easily confused because diagnostic tests are always regarding a single issue.

Screening tests can, and often do, include multiple issues of concern. This is because there is no known problems and there are often several potential issues of concern.

The trick is that screening tests can be single issue.

Don't believe me. Go to the doctor for you annual physical and he may do a rubber-glove screening test (this-won't-hurt-a-bit/this-will-only-take-a-minute) regarding the single issue of enlarged prostate? It is not a diagnostic test. It is a single issue test. It does not provide diagnostic accuracy. Like all screening tests, it only alerts us to the possible presence of a problem for which we need more information before making a diagnosis.

There are other examples of single issue screening tests.

I believe this is the reason the NAS/NRC report used the term event-specific instead of single-issue in the 2003 report. Event-specific reminds us that a diagnostic test is defined by a known problem or known event.

Screening test accuracy is expected to be different than diagnostic accuracy.

First, how do you measure accuracy?

If you are not using a computer algorithm like OSS-3 or a manual scoring model like ESS then you do not have an inferential statistical classifier to describe your test accuracy. ( and some agorithms do not even provide a statistical classifier). In this case you are left to describe accuracy using Bayesian statistics for which you need to have a reasonable estimation of the base rate.

With event-specific tests you have an assumed moderate to high base rate, which we sometimes arbitrarily set at .5, else you would not be testing the examinee.

With screening tests you must estimate your base rate based on your a priori knowledge of the incidence rate of the behavioral issue for the population to which your examinee belongs.

Then, complicate this further by the fact that test sensitivity is sometimes considered the highest priority for screening test designs. On the other hand, high test specificity or balanced sensitivity and specificity or sometimes considered higher priority for diagnostic tests for which the test results may be used to formulate some form of action that will affect the future of the test subject. The circumstances of a known problem, leading to a diagnostic test, often mean it will be unethical to neglect to take action in response to a positive test result (e.g., neglecting medical care after a medical diagnosis).

Short answer:

no, they are not expected to be the same.

in fact, multiple issue screening tests are probably desirable, because they provide better sensitivity and fewer false-negative errors.

good question.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 05-30-2011 03:37 AM            

I am talking about accuracy of the tests is based on validation studies.

As I understand.
1. Event-specific single-issue test is diagnostic test.
2. Standalone screening single-issue test is not diagnostic test. For example, some fidelity tests are not diagnostic tests.

3.1 Screening single-issue test after prior multiple issue testing is diagnostic test because we have known problem (SR in multiple issue test).
or
3.2 Screening single-issue test after prior multiple issue testing is not diagnostic test because we do not have known problem (strongest physiological responses are not always to the question to which the examinee is being deceptive)

What variant is right, 3.1 or 3.2?

As I know Utah ZCT event-specific single-issue test has accuracy for Deceptive cases - 92% and for truthful cases 89% (Krapohl, 2006).

Can we say that this test would has such equal accuracies in the variants 1 and 3.1 and this test would has lower accuracies in the variants 2 and 3.2?

posted 05-31-2011 06:49 AM            

I answered your question.

a single issue screening test is a different animal than an event specific diagnostic test.

You will not get scientists to agree that a failed screening polygraph represents a known problem for which a subsequent polygraph can be diagnostic.

You are still engaged in a fishing trip regarding an unknown problem.

The scientific problem here is that of independence of method. You are attempting to using the polygraph to both allege and prove the problem.

Bad form.

Successive hurdles is a good idea. But your second test is still a screening test.

We are going to get ourselves into un-win-able arguments if we try to pretend that the polygraph can provide that kind of diagnostic precision regarding amorphous issues.

In medicine it is different, because they have tangible things to eventually find. Cells, and antibodies, bacterium, and viri - things they can actually squish and put under a microscope to take a picture of.

The business of lying is much more intangible.

A good scientific screening and diagnostic testing comes down to this: if we use the polygraph to identify the problem we need extrapolygraphic evidence to confirm it.

If you have extrapolygraphic evidence already, then polygraph may become a form of additional confirmation. This is at the core of the scientific issues regarding the empirical meaning of the test result.

In other words: what does it mean if all you have is the test result?

If you answered "nothing," then I will suggest you evaluate whether you believe the polygraph works.

If you know that the evidence says the polygraph does work, then we will be insuring our future by showing people how to take the result seriously - and that will require we interpret the results correctly.

We know that the polygraph is very accurate and not perfect. If you run the test properly your results will be accurate most of the time. However, it is conceivable that not all results are accurate. On occasion, it might be wrong. It is also conceivable that on occasion you can do everything right and the test might still be wrong. (I know there are some concrete minded people who cannot handle this fact of reality and might prefer to pretend.) Not often, but it is conceivable if the test is not perfect. there is no perfect test, and there is no test that will work with everyone.

So, if there is something about the circumstances or the examinee that causes the polygraph result to be incorrect, you have those same conditions when you do another polygraph.

Paired testing is different. With paired testing you have two examiners and two examinees. The probability of an error is very low when the two results agree.

We know that a single issue test is more accurate, in terms of balanced sensitivity and specificity, compared to a multi-issue screening test. Multi-issue tests are designed to have high sensitivity, and sometimes do not prioritize specificity.

Like single issue diangostic tests, a single issue screening test will probably provide a better balance of sensitivity to deception and specificity to truthfulness.

Again, unless you are scoring with a method that give you an inferential statistical classifier in the form of a p-value, then your accuracy estimation (.92 or whatever) is a Bayesian classifier which is non-robust against the base-rate. And it is a violation of the scientific principle of independence to use the first polygraph as an indicator of prior probability.

So, no it is not scientifically responsible, nor ethical, to attempt to attribute diagnostic accuracy to single issue screening test.

All we know is that single issue screening tests probably have better balance of sensitivity and specificity. (A multi-issue screening test might have better sensitivity - resulting in fewer false-negative errors).

The more we tighten up on these things and understand the argument, the more likely we can prevail when faced with new challenges to the use of the polygraph. If we take the convenient shortcuts, then we will eventually pay for it. Its happened before.

peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-02-2011 05:06 AM            

quote:

---

in fact, multiple issue screening tests are probably desirable, because they provide better sensitivity and fewer false-negative errors.

---

Maybe I something misunderstand yet, but as I know, for example, TES has accuracy of 83% for deceptive cases, 0% inconclusive (Krapohl, 2006), while:

"Outcomes with the Single Issue approach on Innocent subjects
were 42% correct. 8% incorrect, and 50% inconclusive. Excluding
inconclusives, 83% of these innocent subjects were categorized
correctly. With the subjects who committed one or more crlmes
the Single Issue approach called 82% deceptive to at least one
crime. 8% deceptive to no crimes, and 10% were called
inconclusive. Excluding inconclusives. 91% of the Quilty subjects were classified as deceptive to at least one crime." (Barland, 1989).

Ñan it really be true that single-issue screening test (using in "diagnostic" phase) has accuracy of less than 83% without inconclusives for deceptive cases?

posted 06-03-2011 11:32 AM            

When you write "diagnostic mode" is looks like we are still trying to bootleg (fake) a claim to diagnostic accuracy. It is simply the second phase of a successive hurdles testing strategy, using a single issue technique. It is still a screening exam, not a diagnostic exam. It would be best to remain very clear about this and forgo any use of the word "diagnostic" and any implication that it is a diagnostic test - unless you have reason for concern that is independent of the polygraph.

Also, no test has zero inconclusives. There is no such thing as a perfect test. Krapohl (2006) summarized the research from the Research Division Staff (1995; 1995b). Read the reports carefully, and be realistic. There is no such thing as a perfect test or a test that has no inconclusives. Inconclusive rates might be very low. But if you run enough tests you will find them.

Barland Honts and Barger (1989) was an interesting and important series of studies. It alerted people to a number of issues, and raised some questions. These questions look, to me, like they may have led to the interest in an improved screening technique which led to the TES/DLST. Barland Honts & Barger '89 should not be considered a definitive answer regarding anything (of course no single study can be a definitive answer).

The 1995 reports describe that there were actually 5 inconclusive case in the two studies. Because re-testing is a procedural requirement when the results are inconclusive, the investigators were interested in the accuracy of the results when the scores reached a conclusive decision. Therefore, cases were removed from the reported results when the result from the blind scorer was inconclusive. They also removed some false-positive results - when the examinee made admissions that seemed to account for the error. This looks very odd to some people, but the point was evidently to answer some very specific questions about accuracy for the people that wanted to know.

All studies are flawed in some ways, and all study results are estimates of what might be found in the field with hundreds or thousands of live cases. This is why we require statistical errors of measure or statistical confidence intervals to tell us the range of potential error or bias.

Please be careful.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-05-2011 02:56 AM            

Am I right?

posted 06-05-2011 01:00 PM            

another good question,

this one is simple,

there is no difference...

known allegation
known event
known symptom
known problem

there is simply some prior reason to suspect a the examinee's involvement in a problem

that is - some reason other than membership in a particular group of people who are subject to screening tests

don't make too much of it,

the point is you can have a known event, such as house burned down, and you know there is an event because you have a pile of ashes and a dead guy inside. some evidence seems to implicate your examinee (you would not be testing people at random). your question is this: did the examinee do it?

or you can have an alleged event, such as a forced sexual assault - for which the physical evidence might not exist, or could be accounted for by other confounding causes such as consensual sexual contact - you therefor don't know whether to there is or is not an actual event. you do have a known allegation

both conditions give to some prior (a priori) reason to suspect your examinee is involved in the issue of concern

without the event or allegation you have a screening test: in which case you select examinees for testing simply because they belong to a group of people. you select everyone. or you select people at random. or you select people on a schedule.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)